The present invention relates to a Hamming neural network circuit, and more particularly to an analog integrated circuit of a Hamming neural network which can be fabricated in CMOS (Complementary-Metal-Oxide-Semiconductor) technology.
Artificial neural network models have been studied for many years in the hope of achieving human-like performance in the fields of speech and image recognition. Now the main research method in this field is still using software to simulate some models or realize some algorithms. Although this research method can solve many problems, it is not suitable for some applications requiring real-time processing such as some image and speech recognition applications. The modern VLSI (Very-Large-Scale-Integration) technology has made it possible to fabricate more practicable artificial neural network chip. Using digital logic circuit can not realize really full parallel processing. The artificial neural network realized by analog integrated circuit have full parallel processing capability and other inherent advantages of biological neural networks.
The literature "An Introduction to Computing with Neural Nets", Richard P. Lippmann, IEEE ASSP Magazine, pp. 4-22, April, 1987, provides an introduction to the field of artificial neural networks by reviewing six important neural network models that can be used for pattern classification. As described in Lippmann's literature, these networks are highly parallel building blocks that illustrate neural-network components and design principles and can be used to construct more complex systems. One of the six neural network models is the Hamming network which is a neural network implementation of the optimum classifier for binary patterns corrupted by random noise. The structural model of a feed-forward Hamming network maximum likelihood classifier for binary inputs corrupted by noise is described in FIG. 6 of Lippmann's literature. The Hamming network is a two-layer network, and implements the optimum minimum error classifier when bit errors are random and independent. The lower subnet shown in Lippmann's FIG. 6 calculates N minus the Hamming distance to M exemplar patterns. The upper MAXNET subnet selects that node with the maximum output. All nodes use threshold-logic nonlinearities where it is assumed that the outputs of these nonlinearities never saturate.
The operation of the Hamming network is described in Box 2 of Lippmann's literature. Weights and thresholds are first set in the lower subnet such that the matching scores generated by the outputs of the middle nodes of FIG. 6 are equal to N minus the Hamming distance to the exemplar patterns. These matching scores will range from 0 to the number of elements in the input (N) and are highest for those nodes corresponding to classes with exemplars that best match the input. Thresholds and weights in the MAXNET subnet are fixed. All thresholds are set to zero and weights from each node to itself are 1. Weights between nodes are inhibitory with a value of -.epsilon. where .epsilon.&lt;1/M.
After weights and thresholds have been set, a binary pattern with N elements is presented at the bottom of the Hamming network. It must be presented long enough to allow the matching score outputs of the lower subnet to settle and initialize the output values of the MAXNET. The input is then removed and the MAXNET iterates until the output of only one node is positive. Classification is then complete and the selected class is that corresponding to the node with a positive output.